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Abstract 

Background: When it was initiated in 2001, England's national patient survey programme was one of the first in 
the world and has now been widely emulated in other healthcare systems. The aim of the survey programme was 
to make the National Health Service (NHS) more "patient centred" and more responsive to patient feedback. The 
national inpatient survey has now been running in England annually since 2002 gathering data from over 600,000 
patients. The aim of this study is to investigate how the data have been used and to summarise what has been 
learned about patients' evaluation of care as a result. 

Methods: Two independent researchers systematically gathered all research that included analyses of the English 
national adult inpatient survey data. Journals, databases and relevant websites were searched. Publications prior to 
2002 were excluded. Articles were also identified following consultation with experts. All documents were then 
critically appraised by two co-authors both of whom have a background in statistical analysis. 

Results: We found that the majority of the studies identified were reports produced by organisations contracted 
to gather the data or co-ordinate the data collection and used mainly descriptive statistics. A few articles used the 
survey data for evidence based reporting or linked the survey to other healthcare data. The patient's socio- 
demographic characteristics appeared to influence their evaluation of their care but characteristics of the workforce 
and the. At a national level, the results of the survey have been remarkably stable over time. Only in those areas 
where there have been co-ordinated government-led campaigns, targets and incentives, have improvements been 
shown. The main findings of the review are that while the survey data have been used for different purposes they 
seem to have incited little academic interest. 

Conclusions: The national inpatient survey has been a useful resource for many authors and organisations but the 
full potential inherent in this large, longitudinal publicly available dataset about patients' experiences has not as yet 
been fully exploited. 

This review suggests that the presence of survey results alone is not enough to improve patients' experiences and 
further research is required to understand whether and how the survey can be best used to improve standards of 
care in the NHS. 
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Background 

National patient surveys were first proposed in England 
in a Government policy document The New NHS: Mod- 
ern, Dependable [1]. The survey programme was 
intended to provide "systematic evidence to enable the 
health service to measure itself against the aspirations 
and experience of its users, to compare performance 
across the country, and to look at trends over time." 
Further reference was made to improving patient invol- 
vement in The NHS Plan [2], which mentioned 
"patients' surveys and forums to help services become 
more patient-centred". 

The first survey of adults who had been treated as 
an inpatient in an Acute Hospital in England was con- 
ducted in 2001/02 and surveys have been repeated 
almost annually since. The survey programme has 
generated six consecutive years of data, freely avail- 
able through the Economic and Social Data Service 
(ESDS). 

Government policy documents continue to emphasise 
the importance of a more 'patient-centred' NHS in, for 
example, Equity and Excellence [3] and Liberating the 
NHS: Legislative framework and next steps [4]. The NHS 
Outcomes Framework 2011/2012 [5] includes patient 
experience of care as one of five core domains to be tar- 
geted in the NHS in 2011/12. These are: 

1. Preventing people from dying prematurely, 

2. Enhancing quality of life for people with long-term 
conditions, 

3. Helping people to recover from episodes of ill 
health or following injury, 

4. Ensuring that people have a positive experience of 
care, and 

5. Treating and caring for people in a safe environ- 
ment and protecting them from avoidable harm. 

With patient-centred care high on the political agenda, 
and media reports of contradictions between the survey 
and acute hospital trust inspection results [6], it is vital 
to identify where and how data gathered by this survey 
programme are being used and to what effect. 

This study asks how national adult inpatient survey 
data have been used in research to date to facilitate 
patient-driven improvements in acute trusts in England. 
We also set out to summarise the main findings of the 
programme so far and to identify the potential for 
further research. A preliminary search found no existing 
reviews of research using national adult inpatient survey 
data; this paper is intended to fill that gap. Our research 
questions are: 

1. To what extent has data gathered by the national 
inpatient survey programme been used as the basis of 
empirical analyses and how might the quality of these 
outputs be described? 



2. What are the main conclusions about patients' 
experiences of care in acute hospitals in the NHS that 
can be drawn from the accumulated data obtained by 
the survey programme? 

Questionnaire and Survey Background 

There are numerous patient experience surveys con- 
ducted annually in Europe, the USA and Canada [7]; 
Delnoij (2009) states that these endeavour to obtain 
"detailed reports of what actually happened to patients 
during a hospital stay" not just an overall measure of 
satisfaction. In England, the adult inpatient survey seeks 
to collect the views of recent patients about "how good 
the hospitals are and how they can be improved" [8] as 
well as some socio-demographic information about each 
respondent. 

The two main current uses of the inpatient survey are 
to aid the regulatory functions of the Care Quality Com- 
mission (CQC) and to assist local improvement. The 
patient survey comprises over 20 per cent of the items 
in the CQC's Quality and Risk Profiles (QRP) [9], which 
assess compliance with Essential Standards of Care. It is 
unclear to what extent acute NHS trusts use data 
locally; there appears to be no systematic collection of 
information about how these data are used within hospi- 
tal Trusts. 

Responsibility for the programme began with the 
Department of Health (DoH) and transferred to the 
Commission for Health Improvement (CHI) in 2003. In 
April 2004 authority passed to the Healthcare Commis- 
sion and from October 2008 the survey has been over- 
seen by the CQC. Official results and key findings have 
been reported annually, with trust level reports made 
available from inception [10]. 

The survey was designed by and is conducted through 
the Picker Institute Europe on behalf of the regulatory 
body. Questions for the survey were developed through 
pilot studies that sought to understand the issues that 
are of importance to patients [11]. 

The survey questions are organised under thirteen 
headings that cover the journey of the inpatient from 
arrival at the hospital to discharge. Most ask respon- 
dents to select one option from a set of pre-defined 
responses. Patients may also include their own com- 
ments in a section at the end. 

Using detailed written guidance [12,13], the surveys 
are conducted according to a standard protocol by an 
approved contractor on the trust's behalf or, in a small 
proportion, by the trust itself. For each trust, the sample 
consists of 850 consecutively discharged patients aged 
16 and over. Questionnaires are posted to selected 
patients, and up to two reminders are sent to non- 
responders at 2-weekly intervals. For responders who 
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are unable to read English, a free telephone line is pro- 
vided that aims to connect responders to a translator 
who would complete the questionnaire on their behalf 
over the telephone. 

The highest response rate (64 per cent) was achieved 
in 2002 [14]. Response rates have declined over the 
years to 50 per cent in 2010 [15]. 

Methods 

A preliminary search for systematic reviews that made 
use of national inpatient survey data was conducted 
using the Campbell Collaboration, Evidence for Policy 
and Practice Information (EPPI) and Co-ordinating Cen- 
tre, Database of Abstracts of Reviews (DARE) and Trip 
Databases. The sites were interrogated using variations 
of the term 'systematic review' AND 'inpatient survey' in 
July 2010 and October 2010. The searches indicated 
that no review had been conducted. 

Search strategy 

Websites were selected by their relevance to the NHS, 
the national inpatient survey, following consultation 
with experts in the field [see Acknowledgements] and 
websites of organisations approved by the CQC to con- 
duct the inpatient survey. Journals and databases were 
interrogated and a literature search completed by the 
Royal College of Nursing (RCN). For details of the 
search strategy and the RCN outcome, please see Addi- 
tional file 1. 

Reference manager packages such as End Note and 
Mendeley were tested for suitability for managing the 
research collection process. Mendeley [16] was selected 
as the authors found that it had some advantages: 

a. The desktop version is free. 

b. The interface between the on-line and desktop ver- 
sions is very easy to use and allows automatic extraction 
of article details. 

c. The identified research (PDFs, word docs etc.) can 
be attached for ease of review. 

d. It allows notes to be put on documents (both inside 
the document itself, and as part of the document detail, 
i.e. author, title, year), again to assist the review process 
and include reminders of justifications for article exclu- 
sions/inclusions. 

e. It is easy to share among co-authors. 

A scoping search was conducted in July 2010, followed 
by a full systematic investigation conducted by two inde- 
pendent researchers in October. The search was also 
updated by one researcher in January 2011. Search 
results were screened by the researchers based on title 
and date of publication and if these suggested that the 
paper fit the inclusion criteria, the abstract was read. 
Full papers were obtained for all unique research articles 
identified in the search as well as a group of papers that 



were recommended by experts in the area. These were 
read by the two reviewers who each formed a view as to 
whether they met the inclusion criteria. Where they dis- 
agreed, a third reader was consulted. Papers were read 
in full and further exclusions made. 

Search terms 

To ensure a consistent and replicable approach, PICO 
(population, intervention, comparison and outcome) 
methodology was adopted to create an extensive list of 
search terms, highlighting differences in phraseology, spel- 
ling and acronyms. The search terms were combined with 
lessons learnt from performing the preliminary search and 
adapted according to search engine capability. For a break- 
down of search terms and mechanisms used to identify 
papers for review, please see Additional file 2. 

Eligibility criteria 

Identified papers made reference to a minimum of one 
year of national inpatient survey data. Papers reporting 
response rates only were excluded. Research comprising 
direct analysis of raw figures, or extracts from annual 
regulator-led findings, were both included. Articles 
reporting results at trust level that included national 
comparisons were accepted. Papers reporting results for 
one trust only, with no national referencing, were 
excluded. Research that made use of models which used 
the inpatient survey as one of numerous indicators, such 
as the star ratings system or DoH toolkit for analysts 
[17], were excluded as the inpatient survey was not the 
main focus. Papers that discussed the survey in general 
without including or analysing data were excluded, as 
were articles based on another paper without reporting 
independent analyses. 

No exclusions were applied regarding ethnicity, gen- 
der, language [18] or country of origin. However, the 
search was limited to publications no earlier than Janu- 
ary 2002 because the first survey was conducted in 
2001/02. Research based solely on another survey, such 
as United States' National Inpatient Survey (NIS) or 
Patient Experiences Questionnaire, were excluded. Grey 
literature was beyond the scope of the search and conse- 
quently not included. 

Controlling for bias 

To ensure the search was comprehensive and to reduce 
the impact of researcher selection bias, two research fel- 
lows systematically and independently conducted 
searches of the literature and their findings were amal- 
gamated. A third reader assisted with the process of 
deciding which papers should be included. Search terms, 
sources and eligibility criteria were pre-agreed to ensure 
consistency and make certain the requirements were 
fully understood. A wide variety of research studies were 
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included and experts were consulted [see Acknowledge- 
ments], to ensure a fair and wide search. 

Results 

Identified papers were amalgamated and duplicates 
removed, as outlined in Figure 1. 

64 papers were selected as meeting initial review criteria. 
23 were later removed following screening of whole docu- 
ments. The 41 papers included for final review were bro- 
ken down into the following five categories: 

♦ Annual survey reports [9] 

♦ Evidence based articles and reports [13] 

♦ Multiple survey comparison reports [9] 

♦ Ethnicity, age and patient experience [3] 

♦ Sociological studies [7] 

Excluded papers 

Of the 23 papers excluded, five addressed the surveys' 
impact in general without analysing survey data, these 
detailed: 



• Why it is important to measure patients' experi- 
ences, what aspects you should measure, and methods 
of collecting information [19]; 

• A guide to assist trust boards and stakeholders to 
determine the most appropriate methods of measuring 
patients' experience; including feedback mechanisms 
[20]; 

• The CQC's performance indicator calculation meth- 
odology for patients' experience, including categorisa- 
tion, domain structure and scoring [21]; 

• NHS trust representatives' perceptions of the survey 
programme, what would encourage greater use and cur- 
rent obstacles against adoption [22], and 

• Using the inpatient survey results, as one of many 
factors, when choosing a hospital for elective admission 
[23]. 

Four articles discussed the findings of other papers 
that analysed data but did not report independent ana- 
lyses of survey data [24-27]. Four more papers made no 
reference to the acute trusts adult inpatient survey: 



Figure 1 : Search results 
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♦ Two discussed other surveys and were picked up by 
a search of 'inpatient' and 'survey' separately [28] and 
'inpatient survey' (of orthopaedic patients) [29]; 

♦ Palmer [30] was identified by the Hospital Consumer 
Assessment of Health Plans (HCACPS) 'national inpati- 
ent survey', and 

♦ Sheikh [31] responded to an article that analysed a 
Swiss 'inpatient survey'. 

Shipton et al. [32], Stevens et al. [33] and Davies et al. 
[34] also did not use data, although authors did adopt 
the star ratings system as a measure of performance. 
Peytremann-Bridevaux et al. [35] used an adaptation of 
the Picker inpatient survey questionnaire for a study in 
University Hospital of Geneva, but did not make use of 
the English national inpatient survey data. A paper by 
Crow et al. [36] used the survey data for one trust but 
did not refer to national data. 

Two reports discussed ethnicity in terms of response 
rates and increasing representation. One reported the 
demographic profile of responders, but not their 
reported experiences of care [37]. The second reviewed 
literature outlining strategies to increase the response 
rate of minority groups [38]. Likewise, Jumaa [39] 
referred to the response rate of the inpatient survey 
with no experience data or reference to year. 

Finally, two articles outlined local projects that made 
use of trust level survey results to improve local services 
and measure change. The first reviewed results along- 
side Patient Advice and Liaison Services (PALS) [40]. 
The second used survey results as a measure for the 
success of pain improvement techniques [41]. 

Annual survey reports 

Annual key finding reports were identified for 2002 [14], 
2004 [42], 2005 [43], 2006 [44], 2007 [45] and 2008 [46]. 
The Picker Institute Europe produced all of these on 
behalf of the regulatory body responsible for the survey 
at the time. In 2008, the key findings paper was accom- 
panied by an historical comparisons document [47] and 
a web-based information page, which included a briefing 
report and trust level reports. In 2009 the historical 
comparisons document [48] and website [15] replaced 
the key findings report. 

The first report in 2002 covered every question in the 
survey through descriptive explanations, bar charts and 
tables. Respondent demographic characteristics such as 
age, ethnicity and health status, were reported alongside 
Trust characteristics such as size and location. Each 
demographic variable was related to the questions. 
Older, male and white patients reported better experi- 
ences. Patients with poorer self-rated health status and 
those living in London reported less favourable experi- 
ences. Overall, 74 per cent rated the care they received 
as excellent (38 per cent) or very good (36 per cent). 



In 2004, the report was reduced to descriptive 
national findings for key domains such as Patient Care 
and Treatment and Leaving Hospital. Results for each 
question were tabulated and also cross-tabulated to 
show percentage change between 2002 and 2004. All 
reported findings were statistically significant. Those 
with poorer health were more negative about the care 
they had received. Black, minority and ethnic (BME) 
groups, particularly people of South Asian origin, were 
more negative, though the differences were quite small 
relative to other factors. As in the previous survey, 
older people, men, elective patients, people living out- 
side London and those who were treated in specialist 
Hospital Trusts reported more positive experiences. 
The most significant factor in explaining variations in 
patients' experiences was self-reported health status; 
those who reported their own health to be good, very 
good or excellent tended to report a more positive 
experience of care. 

The 2005 report was more comprehensive than pre- 
vious years. Cross-tabulated results included demo- 
graphic data on respondents and non-respondents, and 
displayed 'all respondents' beside '18 and over' to allow 
for comparison to 2004. Only significant findings were 
included and the Bonferroni correction method 
applied. Due to the "differences in the sample" (16 and 
17 year olds not included), results for the 2004 survey 
were not included in the annual reports from 2006 
onwards [44]. 

The 2006 to 2008 surveys followed the same structure 
as 2005, with increasingly sophisticated analyses. Com- 
parisons between years were tested for significance 
using Z-tests and the Bonferroni method. From 2008, 
tabulated results for the relevant years were simplified. 
Results were presented in full for all response options, 
with no further grouping or comparisons. Significant 
changes over time were sought by comparing the last 
two years (2007 and 2008) with the survey results from 
the first year (2002). 

In 2004 selected 'free text' comments were included to 
support key findings. It was not until 2007 that the full 
free text comments were included in an additional sum- 
mary report [49] which showed that nearly 60 per cent 
of respondents wrote at least one comment suggesting 
that these free text comments are also a rich source of 
data which have not been analysed in any other way 
apart from this one study conducted by the Picker Insti- 
tute Europe in 2007. 

Evidence based articles and reports 

A number of reports and articles identified through the 
review referenced the inpatient survey as supporting evi- 
dence. The papers below quoted figures from the annual 
survey reports: 
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♦ The Patients Association (2009) used the overall care 
rating to highlight poor treatment of elderly relatives 
[50], 

♦ Vere-Jones (2006) quoted 20% of patients not receiv- 
ing help with their food as part of "Cash-strapped 
trusts" saving funds through catering cut backs [51], 

. Handley (2009) wrote about Walsall Hospital NHS 
Trust, who used discharge results to provide follow up 
calls to patients as part of a pilot study [52], 

♦ Pickersgill (2010) discussed nurses influence over 
survey questions, such as patients' finding someone to 
talk to about their concerns [53], 

♦ Staines (2008) examined the influence of the media 
on public perception, such as results for cleanliness fol- 
lowing reported outbreaks of c.Difficile [54], 

♦ Swain (2007) highlighted the benefits of working 
with patients; that the specific nature of survey ques- 
tions provided precise guidance of where services might 
be going wrong [55], 

♦ The Healthcare Commission (2006) used the survey 
to review the management of the admissions process for 
inpatients [56], 

♦ The Healthcare Commission (2007) reported on spe- 
cific questions, such as privacy and talking in front of 
the patient as if they were not there, to highlight "dig- 
nity in care for older people while in hospital" [57] 

♦ Fitzpatrick et al., (2005) wrote about health and 
health care inequalities in England based on demo- 
graphic information collected [58], and 

♦ The NHS Confederation (2010) used the inpatient 
survey as a measure for targeted improvement pilots at 
local trusts [59]. 

One article solely reported on the findings of the inpa- 
tient survey for 2009 [60], highlighting differences from 
year to year and concludedthat while some improve- 
ments have been made, there are "still key areas of con- 
cern" such as 45 per cent of patients reporting a lack of 
information regarding potential side effects of new 
medication. 

The authors of one study adapted the Picker Institute 
questionnaire to conduct an in-house pilot to test how a 
hospice was performing when compared to national 
hospital results for 2007 [61]. Unfortunately the sample 
size was too small to draw any conclusions. 

Finally, an internal publication produced by the 
Department of Health used factor analysis to determine 
a model that linked patient experience to satisfaction, 
based on 2001/02 inpatient survey data [62]. The paper 
also reported analysis produced by the University of 
Sheffield using inpatient survey data for 2003/04, which 
sought to explain the variation between patients' experi- 
ences in the five core domains identified in the NHS 
Outcomes Framework 2011/2012 [5] as set out above. 
In sum, there is a substantial body of work that has 



made use of the national inpatient survey. The fact that 
so many authors have drawn on these data for diverse 
purposes does suggest that it is an important national 
resource and this should be considered in debates about 
the future of the survey programme. 

Multiple survey comparison reports 

Nine papers made use of the national inpatient survey 
to both enhance an argument and to draw conclusions 
with other national NHS surveys. One such paper, A 
High-performing NHS? [63], reviewed the evidence on 
whether increased Government NHS funding had 
improved eight core domains from 1997 to 2010. The 
domains comprised access, patient safety, promoting 
health and managing long-term conditions, clinical 
effectiveness, patient experience, equity, efficiency and 
accountability. Key points, the situation in 1997, pro- 
gress since then and future plans were discussed for 
each domain. The study included inpatient survey 
results for 2008 with healthcare surveys from different 
years and specialties, such as mental health. The inpati- 
ent survey results were reported briefly; patients ability 
to choose the hospital where they were treated had 
increased from 28 per cent to 33 per cent from 2007 to 
2008, 21 per cent of patients reported not being given 
enough information about their condition or treatment, 
and there had been a slight increase in patients report- 
ing that they had been asked their views on quality of 
care while in hospital (6 per cent to 9 per cent from 
2002 to 2008). Similarly, the NHS Confederation's Lost 
in translation [64] briefly reported inpatient survey 
results for 2005. They concluded that patients report 
"high levels of satisfaction with the NHS and care they 
[patients] receive", with 92 per cent describing their care 
as good, very good or excellent (the other options being 
'fair' or 'poor'). The findings were used to support the 
contention that patients' evaluation of their care is more 
positive than the predominantly negative perceptions 
communicated by the media and the general public. 

Two reports provided a national overview of the NHS. 
One gave a snap shot of "patient and public expecta- 
tions, experiences and evaluations" [65]. The Chartbook 
concluded that though some areas have improved, such 
as access to care, important variables such as patient 
engagement in decision making have not. The second 
report briefly discussed the inpatient survey from 2002 
to 2007 as part of An Economic Health Check providing 
an overview of the NHS [66]. The report described a 
"largely static picture" in five key areas: access and wait- 
ing; safe, high quality, coordinated care; better informa- 
tion, more choice; building closer relationships and 
clean, friendly, comfortable place to be. 

A further six reports were produced by the Healthcare 
Commission or Picker Institute Europe. The first report, 
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State of Healthcare 2007 [67], highlighted areas that had 
improved as well as areas where further improvement 
was needed. The Picker Institute Europe sought to 
answer the question "Is the NHS getting better or 
worse?" in an overview of 15 national surveys between 
1998 and 2004 [68]. One of the reported conclusions 
was that government targets and coordinated action 
have had a positive effect on patients' experience. A 
similar report examining trends of inpatients' experi- 
ences between 2002 and 2004 [69] also concluded that 
coordinated action facilitated improvement. A report 
from the Picker Institute, Is the NHS becoming more 
patient centred? [70], covered 26 surveys from 2002 to 
2007, including inpatient data from 2005 and 2006 and 
also concluded that "many aspects of care targeted by 
the government have improved significantly". The report 
also highlighted significant improvements, such as being 
treated with dignity and respect, and areas of continued 
concern, such as cleanliness and food. 

Finally, the Healthcare Commission produced a report 
that analysed the national patient survey programme for 
2003/04 [71]. The report covered inpatient survey data 
from 2004 and used multiple linear regression analysis 
to show that self-reported health was the most signifi- 
cant variable for explaining variations in patients' experi- 
ence. The analysis also revealed that elective patients, 
those admitted to specialist trusts, those admitted out- 
side London, older people and men were likely to 
respond more positively which supports the conclusion 
that there are important socio-demographic variables 
that need to be taken into account when interpreting 
the survey results. 

Ethnicity, age and patient experience 

One study investigated whether self-reported ethnic 
group influenced patient experiences [72] using inpati- 
ent survey data from 2006 as well as other healthcare 
surveys such as the outpatient survey (2004/05). The 
authors caution against a too literal interpretation of 
their findings, however, their analysis suggested that 
compared to the largest category, "white British", most 
BME groups in England were more negative about their 
experiences as patients, particularly in responses to 
questions about access and waiting times, involvement 
in decisions about their care and treatment, and the 
quality of information given to them. They were also 
more likely to say that staff talked about them "as if 
they were not there". Only those respondents who 
described themselves as "white Irish" were more likely 
to report more positive experiences than the baseline 
category of "white British". Ethnic differences were least 
marked in the inpatient survey compared to surveys 
from different healthcare settings, such as the commu- 
nity. This study was updated in 2009 [73] and once 



again, the results showed that apart from white Irish, 
BME populations were less positive about their experi- 
ences of care, particularly with regard to access and 
waiting and relationships with staff. 

In 2001, the UK government declared its intention to 
"root out" age discrimination in the health service 
through the National Services Framework for Older Peo- 
ple [74]. Lievesley et al. [75] used inpatient survey data 
for 2004 along with other healthcare surveys to assess 
whether ageism and age discrimination were still appar- 
ent in the NHS. The report revealed that the oldest (81 
+) and youngest (16-35) hospital patients were most 
likely to feel that doctors and nurses talked about them 
"as if they are not there". Patients aged over 81 were 
also less likely than those aged between 51 and 80 to 
rate their overall care as "excellent". The authors quoted 
results from the 2006 inpatient survey as described in 
the National Services Framework for Older People: older 
patients were particularly affected by hospital manage- 
ment issues, such as privacy, continence management, 
single sex accommodation and provision of nutritious 
food. A number of reports have documented that older 
patients tend to be less critical [57,71], making the more 
negative evaluation of the very oldest patients in this 
report all the more striking. 

Sociological studies 

Several studies have attempted more complex and theo- 
retically informed analyses using the national adult inpa- 
tient survey. These include studies that correlate other 
variables with overall evaluation of care, studies that 
compare the inpatients survey results to the NHS staff 
survey and one study that investigated how workforce 
and community characteristics affect levels of "civility" 
in acute Trusts. 

A report by Ipsos MORI, Frontiers of performance in 
the NHS II [76], analysed inpatient survey data from 
2006 along with survey data gathered from patients 
treated in the community from 2005. The variables 
correlating most strongly with "overall rating of care" 
were with being treated with dignity and respect, 
cleanliness of the room and ward, and being included 
in treatment decisions. The "relative importance" of 
these factors was calculated, with dignity and respect 
being the most important (59 per cent), followed by 
being involved in decisions (28 per cent), and then 
cleanliness (13 per cent). The authors used stepwise 
regression to identify these "most important" factors, 
but did not address the well-known criticisms of the 
stepwise approach [77]. Among these problems are 
that standard errors and /J-values are biased toward 
zero and parameter estimates are biased away from 
zero. These problems are known to be especially severe 
in the presence of collinearity. The method by which 
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they calculated the "percentage importance" of each 
factor was also not explained. 

Ipsos MORI also investigated the association between 
inpatient satisfaction and characteristics of the geogra- 
phical area in which the hospital was located. The 
authors based their characterisation of a trust's "local 
population" on the closest 2,100 output areas (as defined 
by the Census [78]) and found only relatively weak asso- 
ciations between patients' satisfaction and ethnic divi- 
sions and age, and no association with the index of 
multiple deprivation. They identified two external fac- 
tors as the most important, again using stepwise regres- 
sion. These were whether the trust was in London and 
the percentage of the local population aged under 16. 
They then compared "predicted" (using these two fac- 
tors in a regression) and observed ratings of overall 
inpatient care. 

Doyle et al. [79] conducted a similar analysis to 
demonstrate a way that trusts could identify the "key 
drivers" of quality. The paper compared strength of 
association between overall satisfaction and various 
components by generating 3x2 contingency tables, 
with three categories of overall satisfaction, Positive, 
Fair, or Poor. Data from 2006 and 2007 were analysed 
separately. Chi-square tests were then performed on the 
contingency tables to test for a statistically significant 
association, and finally Cramer's V was calculated to 
assess the bivariate "strength of association" (though it 
is noted that since one of the variables has two cate- 
gories, it is actually the phi statistic [see Figure 2]). No 
correction was made for the multiple pairwise compari- 
sons that were carried out. The analysis was at the 
patient level, but no allowance was made for clustering 
within trusts. The authors report that being treated with 
respect and dignity was the most important component 
of overall satisfaction, followed by how well doctors and 
nurses worked together, and confidence and trust in 
nurses. 

Sizmur et al. [80] investigated whether patients' con- 
sidered certain aspects of their experience of care to be 
more important than others or if all were felt to be 
equal. The author tested which elements of the 2008 
inpatient survey were most highly correlated with 




Figure 2 Phi statistic. 



'overall satisfaction' as measured by "Overall, how would 
you rate the care you received?" To ascertain the survey 
items with the strongest relationship to overall satisfac- 
tion, basic correlation analyses were applied to indivi- 
dual questions and overall satisfaction, at the patient 
level. Composite scores were created that revealed con- 
sistency and coordination of care, nurses, and patient 
involvement to be most strongly correlated with overall 
satisfaction. Multivariate stepwise regression was applied 
to account for confounding factors, such as gender or 
admission to hospital, and to identify specific factors 
that might predict satisfaction independently. The 
method resulted in similar predictors of overall satisfac- 
tion as the previous method, such as consistency and 
coordination of care. The author also performed analysis 
at the trust level, but suggested that as the variance 
accounted for implied high levels of multicollinearity (95 
per cent), the results were "likely to be unstable". 

Two papers investigated whether there was a homol- 
ogy between the responses on the staff and the inpatient 
surveys. Raleigh et al. [81] used twenty eight staff survey 
items as explanatory variables in regressions involving 
four inpatient survey responses from 2006 data. Results 
were aggregated to the trust level to allow the two data 
sets to be merged. Dummy variables for whether the 
trust was inside or outside London and trust type (gen- 
eral, teaching or specialist trust) were included in all 
these regressions. The selection of dependent variables 
was carried out using stepwise regression. The authors 
acknowledge some of these problems in their discussion, 
but do not explain why they did not use alternative, 
more robust methods such as LASSO or LAR [82]. 
They argue that their results "...show some significant 
patterns, indicating that the safety and quality of ser- 
vices and patient experience could improve if trusts 
acted on feedback from their staff." 

Dawson (2009) asked whether the experience of staff 
working in the NHS was reflected in patients' experi- 
ence of care [83]. Using data from 2007 to test for cor- 
relations between trust level aggregated inpatient and 
staff survey items he found a staggering 12,214 bivariate 
correlations of which 56 per cent were statistically sig- 
nificant. No level of significance was given, and there 
was no evidence of Bonferroni or alternative multiple 
test correction (a Bonferroni correction would imply a 
level of significance of 0.05/12214 « .000004, and conse- 
quently fewer statistically significant correlations). Some 
of the key findings were intuitively plausible, for exam- 
ple, high levels of bullying, harassment and abuse 
against staff by outsiders were related to many negative 
patient experiences. Similarly, inpatients' perceptions of 
the adequacy of staffing levels and the amount of dignity 
and respect with which they were treated were corre- 
lated with employee's feelings of work pressure and 



DeCourcy et al. BMC Health Services Research 2012, 12:71 
http://www.biomedcentral.com/1472-6963/12/71 



Page 9 of 12 



staffing levels. However, some findings lacked any theo- 
retical or intuitive support, such as higher numbers of 
staff who have had health and safety training leading to 
more patients perceiving staff as conscientious and 
available. This suggests that a stronger theoretical fra- 
mework could be a useful guide to future analyses. 

The Healthcare Commission's Acute Hospital Portfolio 
Review: Ward Staffing [84] also linked inpatient and 
staff survey data and evaluated changes in ward staffing 
since the Audit Commission's first investigations in 
2000/01 [85]. They reported a very strong relationship 
between the use of temporary "bank" and agency staff, 
indicating high levels of vacancies and low levels of 
patient satisfaction. London hospitals had high vacancy 
rates and used temporary staff more frequently than 
hospitals outside the capital. This relationship was so 
strong that it made it difficult to investigate other staff 
variables related to patients' evaluations of care so sepa- 
rate analyses had to be conducted on Trusts inside and 
outside London. These analyses showed that higher pro- 
portions of registered, more experienced and skilled 
staff, greater numbers of staff who were satisfied with 
their jobs and had lower expressed intention to leave 
their current job were related to better patient evalua- 
tions of care. Consistent with the Audit Commission's 
earlier findings, the Healthcare Commission found no 
relationship between the total number of nurses or total 
expenditure and patient satisfaction. Together, these 
findings suggest that to improve patients' experiences, 
hospital trusts should invest in a richer skill mix (more 
experienced and skilled nurses), rather than just increas- 
ing the number of nursing staff. Although these results 
could be important in improving the quality of care in 
the NHS, it was difficult to critically appraise the 
research conducted by the Audit Commission as the 
report did not give detailed information about the data 
used and the ways in which they were analysed. Their 
report was originally backed up by a detailed document 
explaining the statistical analyses but a Freedom of 
Information (FOI) request to the CQC confirmed that 
this paper is no longer available. 

The final article in this review used inpatient survey 
data to derive a trust measure of the "civility" of staff 
towards patients, King et al. [86]. The measure was a 
14-item scale, using questions such as being treated 
with dignity and respect. Additional data sources were 
used to calculate the ethnic diversity of the medical and 
nursing workforce in the trust, the similarity of the eth- 
nic makeup of the workforce and the local population, 
and the trust's performance. Civility was found to be 
negatively associated with ethnic diversity in the medical 
and nursing workforce. Low civility was in turn asso- 
ciated with poor trust performance. On the other hand, 
similarity of the ethnic makeup of workforce and local 



community was associated with increased "civility" and 
hence with better trust performance. The study had a 
strong theoretical framework from which hypotheses 
were derived and tested. A number of different datasets 
were used and data were analysed using appropriate sta- 
tistics. One problem that remains is that it was not 
entirely clear that the 14-item scale actually measures 
civility and this should be tested in future research. 

Discussion 

After a systematic search of the published literature, we 
identified a number of reports and publications that 
were based on analyses of national adult inpatient sur- 
vey data conducted in England from 2002 to 2009. 
These explored the role of ethnicity, age and factors 
that influence patients' experience of care and assessed 
the association between staff and community character- 
istics on patients' evaluation of care. 

Some reports used a single year of data and others 
used multiple years providing historical comparisons or 
trend analysis. Many of these were compiled by the 
organisations involved with co-ordinating or collecting 
the survey data. Results were mostly displayed descrip- 
tively, with more recent papers applying more advanced 
analytical techniques. Many of the identified papers did 
not perform original analyses and used reported out- 
comes from official annual survey reports. Sometimes 
the data were used in questionable ways, such as com- 
paring different datasets as if they were alike and using 
patient level or trust aggregated data. The adopted 
methodologies varied greatly and the choice of statistical 
techniques was not always explained. 

The question "Overall, how would you rate the care 
you received?" was used by many papers to gauge the 
importance patients' placed on different aspects of their 
care. However, patient surveys were emphatically not 
designed as satisfaction surveys. Previous research has 
shown that patients tend to be reluctant to make nega- 
tive comments about their overall experience and satis- 
faction measures provide an unreliable measure of 
quality [87]. Asking patients to report in detail on "what 
happened" is much more useful. There is consequently 
a real need for the development of a summary score 
which combines all of the questions included in one 
survey to create a dependent variable that better repre- 
sents the breadth of topics covered. 

One of the findings from the national inpatient surveys 
was that some patient characteristics relate in a statisti- 
cally significant way to their evaluation of care, for exam- 
ple, age, gender, ethnicity, their evaluation of their own 
health status and whether or not they were admitted as 
an emergency. Findings also report that Trusts outside of 
London tend to receive more positive responses than 
those inside the capital [71]. A recent paper suggested 
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that ethnic diversity, both in the Trust and the local com- 
munity, can also shape patients' experiences [86]. Studies 
linking data from the staff surveys to the inpatient survey 
demonstrate a relationship between the two, and ward 
staffing characteristics has also been shown to contribute 
to patients' experiences, for example, it is not solely a 
question of staffing volume, but of employing more 
experienced and skilled nurses [84] . 

One of the most important findings from the trend 
analyses is that those areas that show sustained 
improvement such as waiting times, cleanliness, hand- 
washing and mixed sex accommodation are areas which 
have been the focus of national campaigns, using a 
range of mechanisms including targets, incentives and 
penalties to change behaviour [55,68-70]. All too often 
national reports document similar findings from year to 
year, exaggerating small changes by marking them as 
statistically significant (which is a function of the large 
sample size rather than large changes in the percentages 
of responses). Overall, it seems that there was very little 
improvement in patients experience at the aggregate 
level from 2002 to 2009. 

The local use of inpatient survey data for quality 
improvement has not been addressed here. Although the 
search strategy was thorough and systematic regarding 
national usage, it would have required a great increase in 
resources to fully investigate individual NHS trust publi- 
cations or the grey literature, where local quality 
improvement studies might be found. It is known that 
survey data are being used locally, for example alongside 
PALS and steering groups to improve care records [40], 
to manage pain more effectively [41] or to aid local edu- 
cation programmes [88] . However these projects are not 
necessarily officially documented or nationally publicised. 
Investigations into local level usage and trust movement 
against the national average from year-to-year would be 
incredibly beneficial, with the potential to highlight suc- 
cessful schemes for national application. 

The emerging picture is that the inpatient survey is 
not in itself a quality improvement tool. It can monitor 
trends and can provide comparative data but simply 
providing hospitals with patient feedback does not auto- 
matically have a positive effect on quality standards. The 
survey programme has revealed that focusing attention 
on specific areas or devising targets that hospitals are 
expected to attain, can have a beneficial effect on 
patients' reported experiences. The DoH address local 
targeting through the Commissioning for Quality and 
Innovation (CQUIN) payment framework [89,90], which 
incentivises "locally agreed quality improvement 
schemes" through proportioned conditional income. 
Questions from the adult inpatient survey are used as a 
measure in the CQUIN model. 



There is an opportunity for individual academics and 
research teams to produce theoretically informed, metho- 
dologically sound and socially significant research using 
data from the national inpatient survey. As public spend- 
ing is reduced in the in times of economic hardship, 
researchers will increasingly be encouraged to use data 
that are readily available, rather than collect new datasets. 
Funding bodies such as the National Institute for Health 
Services Research Standard Delivery and Organisation 
(NIHR/SDO) [91] are providing opportunities to support 
such projects, which may increase interest in the analysis 
of large national datasets and this survey in particular. 
With many countries around the world basing their 
research on inpatient survey questionnaires [7], there is 
also the potential to conduct comparative analyses using 
international datasets. Such collaborations would enable 
the transfer of knowledge and best practice to improve in 
many healthcare systems. 

Conclusions 

The survey of adult inpatients is now well established in 
the NHS and is emulated in countries around the world. 
The principle that patients must be consulted and that 
their feedback is an important indicator of hospital per- 
formance is now embedded in the NHS. This review 
shows however that information alone does not automa- 
tically translate into improved experience of care. Sus- 
tained improvement tends to be achieved when backed 
by national government campaigns and targets, coupled 
with incentives and penalties. The survey programme 
shows that the NHS has tried over the last decade to 
move away from paternalism towards a focus on patients' 
experiences, but there is still some way to go. Finally, it 
has been shown that there is a need for further investiga- 
tion into local patient-driven improvement schemes and 
that there is a great deal of potential for further analysis 
of national adult inpatient survey data. 
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